What Concepts Does Llama-2 Know?

Large Language Models (LLMs) can implicitly store a lot of information about the world. Llama-2, Meta AI's latest open-source model, is able to correctly complete sentences like "Thelonious Monk plays the ___" (piano), presumably due to having seen this information somewhere before during training. But how is this knowledge organized? Below, we display 6615 concept representations that Llama-2 is able to correctly answer questions about, organized based on ongoing research into how LLMs represent multi-token concepts (things that take several words to express).

Examples of Llama-2 Concepts

Explore Llama-2-7b's Vocabulary Space